FLCD: A Flexible Low Complexity Design of Coded Distributed Computing

We propose a flexible low complexity design (FLCD) of coded distributed computing (CDC) with empirical evaluation on Amazon Elastic Compute Cloud (Amazon EC2). CDC can expedite MapReduce like computation by trading increased map computations to reduce communication load and shuffle time. A main novelty FLCD is utilize the freedom in defining functions develop asymptotic homogeneous systems support varying intermediate values (IV) sizes under general framework. Compared existing designs constant IV sizes, offers greater flexibility adapting network parameters significantly reduces implementation requiring fewer input files groups. The scheme first proposed low-complexity that operate an arbitrary number nodes load. perform evaluations executing TeraSort algorithm EC2 cluster. This time theoretical predictions are validated evaluations. demonstrate 2.0 4.24× speedup compared conventional uncoded MapReduce, 12 52 percent reduction total time, wider range operating schemes.